Skip to content

juliac: log entrypoints and nonstandard typedefs #59108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

juliac: log entrypoints and nonstandard typedefs #59108

wants to merge 31 commits into from

Conversation

timholy
Copy link
Member

@timholy timholy commented Jul 27, 2025

To facilitate creating bindings for other programming languages, this adds logging to the juliac buildscript, optionally outputting a logfile that describes the entrypoints (signatures and return types) and nonstandard typedefs that appear in the entrypoint specifications.

My Makefile skills are rudimentary at best, so please do check that aspect with particular care.

@timholy
juliac: log entrypoints and nonstandard typedefs
ebb3ded 
To facilitate creating bindings for other programming languages, this
adds logging to the `juliac` buildscript, optionally outputting a
logfile that describes the entrypoints (signatures and return types) and
nonstandard typedefs that appear in the entrypoint specifications.
@timholy timholy requested a review from JeffBezanson July 27, 2025 07:36
@timholy timholy added the juliac label Jul 27, 2025
timholy added 2 commits July 27, 2025 04:36
@timholy
minor Makefile improvements
1099753
@timholy
Nicer printing
e90786b 
- remove mangling in favor of native Julia printing
- include argnames in methods
- support and test name argument to ccallable
JeffBezanson
JeffBezanson reviewed Jul 28, 2025
View reviewed changes
JeffBezanson
JeffBezanson reviewed Jul 28, 2025
View reviewed changes
@timholy timholy force-pushed the teh/juliac_headerlogs branch from b983bef to 334f613 Compare July 29, 2025 11:05
@timholy
Copy link
Member Author

timholy commented Jul 29, 2025 •
edited
Loading

OK, this is getting more ambitious but running into issues. I'm now trying to actually use the library and not seeing the names I need:

tim@kenzo:~/src/julia/test/trimming$ make JULIA='../../julia --startup-file=no' BIN=.
../../julia --startup-file=no -t 1 -J /home/tim/src/julia/usr/bin/../lib/julia/sys.so --startup-file=no --history-file=no --output-o hello-o.a --output-incremental=no --strip-ir --strip-metadata --experimental --trim /home/tim/src/julia/usr/bin/../share/julia/juliac/juliac-buildscript.jl /home/tim/src/julia/test/trimming/hello.jl --output-exe true
gcc -march=native -mtune=native -m64 -o hello -Wl,--whole-archive hello-o.a -Wl,--no-whole-archive   -std=gnu11 -I'/home/tim/src/julia/usr/include/julia' -fPIC  -lm -L'/home/tim/src/julia/usr/lib' -Wl,--export-dynamic -L'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib' -ljulia -ljulia-internal
../../julia --startup-file=no -t 1 -J /home/tim/src/julia/usr/bin/../lib/julia/sys.so --startup-file=no --history-file=no --project=/home/tim/src/julia/test/trimming -e "using Pkg; Pkg.instantiate()"
../../julia --startup-file=no -t 1 -J /home/tim/src/julia/usr/bin/../lib/julia/sys.so --startup-file=no --history-file=no --project=/home/tim/src/julia/test/trimming --output-o basic_jll-o.a --output-incremental=no --strip-ir --strip-metadata --experimental --trim /home/tim/src/julia/usr/bin/../share/julia/juliac/juliac-buildscript.jl /home/tim/src/julia/test/trimming/basic_jll.jl --output-exe true
gcc -march=native -mtune=native -m64 -o basic_jll -Wl,--whole-archive basic_jll-o.a -Wl,--no-whole-archive   -std=gnu11 -I'/home/tim/src/julia/usr/include/julia' -fPIC  -lm -L'/home/tim/src/julia/usr/lib' -Wl,--export-dynamic -L'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib' -ljulia -ljulia-internal
../../julia --startup-file=no -t 1 -J /home/tim/src/julia/usr/bin/../lib/julia/sys.so --startup-file=no --history-file=no --output-o libsimple-o.a --output-incremental=no --strip-ir --strip-metadata --experimental --trim /home/tim/src/julia/usr/bin/../share/julia/juliac/juliac-buildscript.jl /home/tim/src/julia/test/trimming/libsimple.jl --output-lib true ./bindinginfo_libsimple.log
gcc -march=native -mtune=native -m64 -shared -o ./libsimple.so ./libsimple-o.a -lm -L'/home/tim/src/julia/usr/lib' -Wl,--export-dynamic -L'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib' -ljulia -ljulia-internal
gcc -march=native -mtune=native -m64 -I. -I/home/tim/src/julia/test/trimming -I/home/tim/src/julia/usr/bin/../lib -o capplication capplication.c -lm -L'/home/tim/src/julia/usr/lib' -Wl,--export-dynamic -L'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib/julia' -Wl,-rpath,'/home/tim/src/julia/usr/lib' -ljulia -ljulia-internal  -Wl,-rpath=. -L. -lsimple   -std=gnu11 -I'/home/tim/src/julia/usr/include/julia' -fPIC
/usr/bin/ld: /tmp/cczir9hy.o: in function `main':
capplication.c:(.text+0x8f): undefined reference to `copyto_and_sum'
/usr/bin/ld: capplication.c:(.text+0xf4): undefined reference to `countsame'
collect2: error: ld returned 1 exit status
make: *** [Makefile:58: capplication] Error 1

tim@kenzo:~/src/julia/test/trimming$ nm -D libsimple.so
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 w __cxa_finalize
                 w __gmon_start__
tim@kenzo:~/src/julia/test/trimming$

@JeffBezanson
Copy link
Member

Try -Wl,--whole-archive libsimple-o.a -Wl,--no-whole-archive.

@timholy
Copy link
Member Author

timholy commented Jul 30, 2025

Thanks. I think this may be ready. One caveat is that I haven't finished the package that parses the logfile to generate the .h file (this one was written manually); it might not hurt to wait to merge this until I know that won't encounter any unexpected road blocks.

@timholy
Copy link
Member Author

timholy commented Jul 30, 2025

I can now verify that it works nicely. https://github.com/JuliaInterop/JuliaLibWrapping.jl

JeffBezanson
JeffBezanson reviewed Jul 30, 2025
View reviewed changes
topolarity
topolarity reviewed Jul 31, 2025
View reviewed changes
Copy link
Member

@topolarity topolarity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the initiative on this, this is lovely!

Some thoughts on the implementation, but I am definitely in favor of the feature 👍

topolarity
topolarity reviewed Jul 31, 2025
View reviewed changes
@timholy timholy mentioned this pull request Aug 1, 2025
Draft
@topolarity
Export ABI metadata in JSON format
c0e1c3e 
With luck, this makes this information easier to consume downstream (and
to expand on in the future).

Also fixes a couple of obscure bugs that would cause mis-behavior in the
presence of recursive types, or which could unintentionally mangle types
in the event of sub-string collisions.
@topolarity topolarity force-pushed the teh/juliac_headerlogs branch from 3dad123 to 4993dea Compare August 1, 2025 14:00
timholy
timholy commented Aug 1, 2025
View reviewed changes
Copy link
Member Author

@timholy timholy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, many thanks!

@topolarity topolarity force-pushed the teh/juliac_headerlogs branch from 7624f80 to d51511a Compare August 1, 2025 21:55
@topolarity topolarity force-pushed the teh/juliac_headerlogs branch 2 times, most recently from 375c436 to 42c8031 Compare August 4, 2025 20:43
@topolarity topolarity force-pushed the teh/juliac_headerlogs branch from 42c8031 to 77b1916 Compare August 4, 2025 21:01
timholy
timholy commented Aug 9, 2025
View reviewed changes
Copy link
Member Author

@timholy timholy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very good. Only small quibbles, then I think this is ready to merge.

contrib/juliac/abi_export.jl
type_ids::IdDict{Any,Int}
end

function escape_string_json(s::AbstractString)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC there isn't a "meaningful" test of this escaping.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we have a number of copies of this now running around (test_print_str_escape_json, json_repr). Can we potentially combine them, or at least put them all in a single file to include as needed, so that we can make sure they all use the same logic?

@timholy
Copy link
Member Author

timholy commented Aug 9, 2025

Oh, and if we don't want the support for recursive types to regress, we should add a test for that too.

@topolarity topolarity force-pushed the teh/juliac_headerlogs branch from 322a9ef to 6161dab Compare August 20, 2025 00:46
vtjnash
vtjnash reviewed Aug 20, 2025
View reviewed changes
vtjnash
vtjnash reviewed Aug 20, 2025
View reviewed changes
@timholy timholy force-pushed the teh/juliac_headerlogs branch from 91be2f9 to d9ea0eb Compare August 31, 2025 10:56
@timholy timholy force-pushed the teh/juliac_headerlogs branch from ec234aa to 7adc37b Compare August 31, 2025 20:34
@timholy
Copy link
Member Author

timholy commented Aug 31, 2025

Not quite sure how to fix the test failure here...I can't even seem to run these tests locally anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vtjnash vtjnash vtjnash left review comments

@JeffBezanson JeffBezanson JeffBezanson left review comments

@gbaraldi gbaraldi gbaraldi left review comments

@topolarity topolarity topolarity left review comments

Assignees
No one assigned
Labels
juliac
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@timholy @JeffBezanson @vtjnash @gbaraldi @topolarity